DCXnJMENT RESraE 



ED 327 553 



TM 015 896 



AUTHOR 
TITLE 

PUB DATE 
NOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Walton, Craig 

Critical Thinking and "The Nation's Report Card": 

1990 Reflections. 

90 

12p. 

Reports * Evaluative/Feasi£)ility (142) 
MFOl/PCOl Plus Postage. 

•Critical Thinlcing; ^Educational Assessment; 
Elementary Secondary Education; Minority Groups; 
National Surveys; Outcomes of Education; ^Reading 
Achievement; *Writing Achievement 

National Assessment of Educational Progress; ^Reading 
Report Card 1971 to 1988 (NAEP); *Writing Report Card 
1984 to 1988 (NAEP) 



ABSTRACT 

The National Assessment of Educational Progress « 
(NAEP) issued two reports in 1990 assessing the nation's current 
educational abilities in reading and writing. The "Reading Report 
Card, 1971-88" presented and analyzed results over an 18-year period 
for ages 9, 13, and 17 years (grades 4, B, and 11, respectively). The 
9-year-olds and 13-year-olds were significantly more likely to reach 
the t>asic difficulty level than were their 1971 predecessors. 
However, the report contained frightening information a£)Out the 
nation's students* chances of functioning competently in today's 
complex society. The writing report, "The Writing Report Card, 
1984-88," which in fact ranges across testing from 1974 to 1988, 
ena£)led the NAEP to conclude that there has t)een no change in writing 
performance over the 14-year period. Relatively more positive than 
the reading report, the writing report still raised questions at)0ut 
the recent emphasis on the writing process and how reading and 
writing reports may he related. Among the positive findings of Doth 
reports were gains for Blacks and Hispanics in some reading areas and 
relative improvement for some minority groups in writing. Overall, 
however, the reports do not provide an encouraging picture of the 
reading and writing competence of the nation's youth. (SLD) 
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Critical Thinking and The Nation's Report Card^ 
1990 Reflections 

I. Introduction. Earlier this year, the National Assessment of Educational Progress 
(NAEP) issued two 1990 reports, assessing our current educational abilities in reading and 
writing.^ I would like to divide my time into two portions - to discuss those findings which 
deal with 4th, 8th and Uth graders' abilities to reason, as seen in those reports, and then 
to suggest what educators and educational policy planners might learn from this for the sake 
of making constructive changes in the years to come. Generally, the report on reading 
contains frightening information from the point of view of our school children's chances of 
functioning competently under the circumstances of our economy, our society's problems and 
our international or global rela:ionships. The report on writing is re! avely more positive, 
although it raises some questions about the recent emphasis on the writing process, and 
about how its findings might be related to those on reading. 

II. The Reading Report Card. 1971-88. Published in January of this year, the reading 
report presents and analyzes results over an eighteen-year period, for ages 9, 13 and 17 
(grades 4, 8 and 11). The NAEP testing presented selections from a variety of kinds of 
reading - plays, poems, biographies, science books, and 'l>ooks about other times and 
places**. The selections were standardized at five levels of difficulty, defined as follows: 

1. Rudimentary: able to follow brief directions, and select words or phrases 
to describe or interpret simple objects; 

2. Basic: able to locate facts from simple paragraphs, and make inferences 
based on short passages; 

3. Intermediate: able to search for, locate and organize information in fairly 



\ Ian V.S. MuUis and Lynn B. Jenkins, The Reading Report Card 197M988, Report 
No. 19-R-Ol (Januaiy, 1990). Princeton: National Assessment of Educational Progress, 
Educational Testing Service; and, Arthur N. Applebee, Judith A, Langer, Ina V.S. MuUis 
and Lynn B. Jenkins, The Writing Report Card. 1984-1988 . Report No. 19-W-Ol (January, 
1990). Princeton: National Assessment of Educational Progress, Educational Testing Service. 
The reports are available by writing to the NAEP, Rosedale Road, Princeton, New Jersey 
08541-0001. 
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long passages, and recognize a paraphrase of such passages; also make inferences or 
generalizations about main ideas; 

4. Adept: able to understand complicated passages, including those in school 
texts; also able to integrate unfamiliar information, react to or explain such materials as a 
whole; and 

5. Advanced: able to extend and restructure information in complicated 
passages, including materials found in professional, business and technical work environ- 
ments; able to understand links between ideas even when those links are not directly 
explained, and to make suitable generalizations even when the text does not include a clear 
introduction or summary. Synthesizing ability is most evident here, as is ability to acquire 
new information. 

Because it covers eighteen years of evaluating reading at these five levels, the report 
also allows us to track generations, since (for example) 1988's 17 year olds were 1984's 13 
year olds and 1980's nine year olds. Where improvements are found, they are seen to carry 
across these years, so that we are can conclude thai positive reforms in the 70's and 80's 
have taken hold and provided measurable, continuing benefits. That is the good news; 1988's 
9- and 13-year olds were significantly more likely to reach the rudimentary or basic level 
than were their 1971 predecessors. 

However, the good news is limited to improved chances that 9- and 13-year olds will 
reach one of those first two of the five levels, and that 17-year olds will reach as far as the 
intermediate level. When we look at performance of 13- and 17-year olds at ihe intennedi- 
ate, adept and advanced levels, we find: 

< a> that 13-yr. olds' chance of reaching the intermediate level are unchanged 
from 1971 through 1988 - about 58%; 17-yr. olds' chances of reaching that level steadily 
improved from 78% in 1971 to 86% in 1988; 

<b> 13-yr. olds' performance at the adept level was virtually unchanged, 
hovering near 10%; 17-yr. olds' performance was also unchanged, staying close to 40% over 
the eighteen year period of NAEP testing; and, finally, 

<c> 13-yr. olds' performance at the advanced level remained unchanged at 
about 2/lOths of one percent, while 17-yr. olds' performance at that level dropped from 
6.6% in 1971 to 4.8% in 1988 - a significant, if small, decline. 

But more to the point, of course, is that about 60% of these young .Americans about 
to enter the 12th grade are not able to understand complicated passages, integrate new 
information into their current thinking, or explain such passages seen as a whole. Worse, 
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95% of these young Americans are not able to search out, locate, summarize and see the 
links between new ideas of the kinds waiting for them in the 1990's world of jobs, using 
money, voting, and forming judgments about adult moral questions. To make matters still 
worse, these findings do not include reference to those who had dropped out of high school 
before or during the Uth grade; that number seems to be holding at about 25% of all those 
who reach the high school-leaving ages of 17-18. 

In this same report, the NAEP investigated home attitudes toward reading - the 
extent to which others at home read daily, monthly or veiy rarely, and the extent to which 
the students themselves read un-assigned materials. From an educator's standpoint it is 
encouraging that 17-yr. olds' proficiencies rose over the 18-yr. period of these studies 
irrespective of reading by others at home. Therefore those with little or no family example 
improved, when they did, because of changes at school About 75% of 17-yr. olds read 
books, magazines or newspapers either daily or weekly, though their proficiency does not 
increase from the weekly to the daily readers. 1988 17-yr. olds were significantly less 
engaged in free reading than their 1984 predecessors. As the report concludes this section, 
the authors note that "it is encouraging to find that reading across the curriculum has 
increased with time, but discouraging that the percentages of students who reported 
frequently reading newspapers, books and magazines remained constant at ages 9 and 13, 
and decreased at age 17. The larger indicators suggest that reading is not a frequent or 
highly valued activity for many students" (44). Another positive note is that the number of 
students having homework has increased, as has the number of hours spent on homework. 

III. The Writing Report Card, 1984-88. In fact the writing report ranges across 
testing from 1974 to 1988, enabling the NAEP to conclude that there has been no change 
in writing performance over that 14-yr. period. Students were presented with a variety of 
writing tasks: 

<a> The informative tasks included writing job application letters, reporting on 
information supplied a out plants, a company, and a "ghost-proof house, and finally, 
"analytic writing from given information", where information about American eating habits 
of one hundred years ago, and from today, was presented for analysis, comparison and the 
drawing of conclusions. 

<b> The persuasive writing tasks included a variety of "convincing others" tasks, 
and also "refuting an opposing position " tasks, for each grade level tested; and, finally, 

<c> imaginrtive writing tasks were evaluated, including one task, about a magic 
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flashlight, for 4th graders both in 1984 and 1988. 

The tests on any of tiiese tasks were scored at five levels, defined as follows: 

0 - not rated: responses which were blank, indecipherable or unrelated to the 

task; 

1 - unsatisfactory: these papers were abbreviated, or circular, or disjointed and 
thus did not manage to address the task; 

2 - minimal: some or all of th elements of the task are used, but it was not 

completed; 

3 - adequate: needed information was used, and was adequately presented so 
as to complete th task assigned; and 

4 - elaborated: students went beyond the essentials, provided additional 
coherence and detail in order to make their point. 

Many students tested were able to perform at minimal levels; relatively few reached 
adequate or elaborated levels. Elementaiy students remained the same from 1974 to 1984, 
and improved on some tasks by 1988, with no declines. Junior high or middle school 
students showed relative stability - no significant declines or improvements. The same was 
true for high school students. 

For our purposes, emphasis must be placed on the adequate or elaborated levels of 
performance because those levels require use of such higher-order reasoning skills as the 
ability to paraphrase, to summarize, to explain links between ideas and to draw inferences 
from various pieces of evidence on the same topic. Eleventh graders reaching 'adequate' or 
above on a job application letter totalled about 68% in both 1984 and 1988 - so that some 
32% were either 'not rated', 'unsatisfactory* or 'minimal'. 

Among the informative tasks was one for 8th and 11th graders designed to be a piece 
of analytical writing about American eating habits - asking students to compare information 
about foods and eating habits in the 188(ys and the *1980's, form generalizations and draw 
conclusions supported by the evidence. On this, the most demanding of reasoning ability 
among the assigned tasks, 78% of 8th graders reached 'minimal' or higher. But only about 
13% reached the top two levels of proficiency, 'adequate' or 'elaborated'. At the Uth grade 
level, about 90% in 1988 reached the minimal level here, but only 14 % reached 'adequate' 
or 'elaborated' (oddly, virtually the same proficiency as with 8th graders). Thus 86% of these 
high school juniors were not able to recognize the needed information when it was 
presented, and then assemble it according to the assigned task. 

Turning to the persuasive writing tasks, those reaching adequate or elaborated levels 
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when "convincing others" totalled 25% for 4th graders, 16% for 8th graders and 28% for 
11th graders. But when asked to refute a stated position, all grade levels did very markedly 
poorer work: 4th graders reaching adequate or elaN)rated totalled 15%; 8th graders 
reached 25% on one assignment and 7% on the other, in 1988; 11th graders reached 
adequate or elaborated levels about 21% of the time on two different assignments of this 
kind, in both 1984 ad 1988 - meaning that about 4 out of 5 high school juniors cannot 
present a rebuttal or refutation on such topics as whether or not their city should build bike 
lanes, or which of two buildings would make for the best recreational facilities. 

Since NAEP introduced an imaginative writing task for the first time in 1984, and 
then only for 4th graders, the results available arc skimpy: a bit less than 9% reached 
adequate or elaborated in 1984, and a statistically significant gain, up to 14%, reached that 
level in 1988. 

overall, the trends in granmiar, punctuation and spelling were good and steady; for 
example, about 15% of 11th graders showed sentence fragment errors at the 90th percentile 
level - meaning that 90% of that group had less than a 15% error level on that skill. 

A sadder finding is that in 1984 and 1988, about half of the students at each grade 
level valued writing skill; this result did not change as they got older. Attitudes about 
writing become more negative as studei . got older: 55% of 4th graders said they liked to 
write, 42% of 8th graders, and 37% of 11th graders. But among 11th graders r/ith a high 
proficiency of writing ability, those seeing themselves as making a frequent use of writing 
in their personal lives jumped from 35 to 46 % between 1984 and 1988. 

Trends in teachers' comments on writing present a fairly positive picture for 4th and 
8th graders, but a mixed picture for 11th graders: the quantity of writing being assigned at 
all levels is rising, so that one might well understand why comment would not remain a 
coastant - students probably benefit from more writing, whether or not it gets detailed 
conmient, and a teacher does not acquire more time for reading papers just because more 
is assigned (nine kinds of comments all showed a decline in frequency between 1984 and 
1938). Teachers' comments on 11th graders' writing were either neutral (such as a note 
from the teacher) or negative (such as marking mistakes or pointing out flaws) in 43-61% 
of students' papers, while positive "omments (such as pointing out strengths, making 
suggestions for next time, or showing an interest in what the student wrote) occur about 
32% of the time. It seems hard to believe that this difference would have no relation to 
11th gra^^rs' opinions of whether or not to write they like to write. 

Overall, two features are noted by the NAEP authors: first, the cause and effect 
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relationships between higher writing proficiency and more positive attitudes about writing 
cannot be resolved - no data tilts the correlation one way or the other; and secondly, after 
ten years of emphasis on the writing process as the most hopeful method of teaching, there 
is no significant improvement in writing performance (though the report does not undertake 
to speculate as to why that might be the case). 

IV. Analysis: Reading, Readers of the NAEFs 1989 study of young adult (-1-26) 
literacy will note the same pattern in the 1990 4th through Uth grade reading report as was 
apparent for young adults: lower order skills such as memorization, single-item match-ups, 
identification or recognition of previously-given items is being drilled to a high degree of 
proficiency by our nation's teachers. But when the task requires thinking two thoughts in 
sequence, doing one job and then a second, comprehending and attending to the meaning 
of a complex passage, or asking oneself what needs to be done, or characterizing a 
relationship or linkage between events, paraphrasing in one's own words, or even 
recognizing a paraphrase of a passage - all higher order reasoning skills - then the large 
bulk of students leaving the Uth grade (and, as we saw last year, reaching in to their later 
twenties) are not able to carry out these kinds of mental activities. Since initial access to 
literacy - at the rudimentary and basic levels - is being accomplished irrespective of family 
example or support, one is led to emphasize this positive achievement as due to conscious 
efforts of teachers and curriculum reformers. 

But - is it equally sensible to suggest that those same educators are not undertaking 
the development of higher-order reasoning skills? My own estimate at this time is that the 
answer must be a qualified 'yes': Curriculum guides, education experts such as Paul Chance 
of Columbia University Teachers College, and educationist articles in this field seem to be 
characterized by two flaws:^ first, they confuse lower with higher-order skills, typically 
describing and giving lesson plans for the development of such rudimentary skills as 
recognition or memory virtually side-by-side with such higher order skills as questioning for 
the missing item, locating cause and effect, organizing inquiry by appropriate steps in ;ome 
definite order, or recognizing or explaining relationships of ideas. Second, when they do 
discuss activities which characterize those higher-order thinking skills, these skills are usually 



I See, for example, Gail Lenning, "What Can the Matter Be? A Study of Critical 
Thinking in High Schools", Master of Arts thesis in Ethics and Policy Studies, University of 
Nevada, Las Vegas (copyright 1990; available on request from Institute for Ethics and Policy 
Studies, UNLV, Las Vegas, NV 89154). 
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jumbled in a g^ab-bag, as if any handful the teacher or student brings up would be a good 
deal, and a lot better than nothing. It is hard to avoid the impression of a mindless 
approach to arts of mindfulness. 

Although this confusion is not the responsibility of the NAEP, one can see even in 
their own characterization of the five levels of reading ability, that whereas a gradation of 
difficulties is apparent, and there seem to me to be gradations of interpretive skills as well 
as of critical or evaluative skills, they are not presented in a pedagogically coherent manner. 
Of course we cannot "teach to" such tests as the NAEP devises, and to try to do so would 
be self-defeating. But at a higher plane, it is not readily apparent how a teacher or 
curriculum plaimer could study the 1990 Reading Report and convert its bad news into a 
set of lesson plans and units which would directly address these issues. And the reason for 
that IS that the education community itself has not reached a working agreement as to what 
thinJdng skills are most generic, such as to give rise to or allow for proliferation into the 
huge variety of species and sub-species one finds in the world of able thinking adults. Nor 
has it been able to agree Oi^ which of the fundamentals must be taught first, in order to get 
to tne second, and then to the third, and so forth - or, indeed, whether reasoning skills are 
sequential and developmental in stages, or else are disconnected, more like separated tools 
hiving no particular relation to each other as to whether or when one uses this one or that 
one, first or last, or both or only one. In view of this confusion, it is no wonder that the 
NAEP results show a near-halting of students' mental capacities' development after reaching 
the second of the five levels explained. Either little or nothing is actually done at these 
higher levels, or it is done with well-intentioned and energetic confusion such as to have no 
discernible result.^ 

V. Analysis: Writing. In one way similar to the 1990 Reading report, a great deal of 
what is currently undertaken in the teaching of writing seems to be working adequately at 
what th NAEP judged to be the minimal level. You might want to study the original for 
details on progress of minorities as compared to Caucasian students, and of males as 
compared to females, etc. In view of the heterogeneity of our country, the data on 
percentages reaching th minimal level in writing must be considered to be a positive 



^. I do address the positive aspect, of which this paper is the negative, in my "Reasoning 
Across the Curriculum" paper, in which I undertake both to present a sequence of six skills 
of critical thinking, and to indicate how they may be well-coached using materials already 
present in K-12 classrooms. 
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achievement of oui schools. 

But, as we :,aw above, there are two findings crying out for explanation: < 1 > why 
do so few reach 'adequate' ["needed information was used, and was adequately presented 
so as to complete the task assigned"], even by the end of the 11th grade ?; and <2> why 
is it that when the assigned task [still under the 'informative' category] required thmking as 
compared to telling, persuading or imagining, that the perfoimance from 11th graders 
reached 'adequate' only for 14% of the students ? 86% of these young people could not 
compare statements about food available, and eating habits, in ways that would present an 
understandable presentation of evidence and draw unstated by implied conclusions - as that 
the lack of refrigeration on the frontier played a major role in what would keep or spoil, and 
therefore of what one could eat when the food was out of season. Since the only data on 
this task stems from 1984 and 1988 testing, it is possible that our reflection on these results, 
together with our reflections on the 1990 Reading Report, might enable us to stimulate and 
guide analytic writing assignments at these secondary levels The same criticism applies here 
as earlier - the students did not know what was asked of them, had no specific instructions 
as to what to do or when, and thus fumbled the ball, or folded up and quit after only 
perfunctory remarks. If we assume, as seems fair, that most of them had not seen such an 
activity in school hitherto, the gross lack of adequacy seems more likely to be due to lack 
of effort by teachers than lack of capacity by students (this seems to be corroborated, 
anecdotally, by my 26 years in college teaching where students typically comment both that 
they never had such work in secondary school, and that they enjoy catching on to it as that 
happens). 

The same holds for failure at writing a rebuttal - if students have no idea what an 
argument is, what evidence looks like when assembled to sup^rt a conclusion, and what 
critical thinking does with questioning the relevance of the evidence or the plausibility of 
the argument, ttien the idea of writing a rebuttal necessarily would seem to them to be 
dropped from outer space, with no forewarning at all. About 4 out of every 5 11th graders 
could not write either of two rebuttals (about buildings useable for recreation, and about 
installing bike lanes in the city). Again my estimate is that these deplorable and frightening 
results could be turned around if secondary education were to conununicate what is meant 
by an argument, and a rebuttal to an argument. The standards used by the NAEP's judges 
in evaluating these student efforts were not overly rigorous, and seem from what I can tell 
to have been fair and judicious, though of course inherently qualitative judgments. 

Finally, to comment on the 1984 '"And 1988 students' characterizations of their 
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writing teachers' attitudes toward their writing: About one-third of the llth graders 
reported at least some positive comment having been included in conmients in general, so 
that two-thirds had only negative or neutral comments. This finding requires r/>te, by itself, 
since it suggests a rigidity or fault-finding mentality to be dominant, and sujh a mentality 
among teachers would not lend itself well to the use, much less the encouragement and 
praising of critical thinking skills. This issue is a red flag; it may denote a 'go by the rules' 
mentality, which would, of course, produce high scores at lower-level, obedience training 
kinds of skills, and would be irrelevant to the exemplifying, coaching or evaluating of any 
gradations of ability with the higher levels of reasoning. As Richard Paul has often 
explained \ critical thinking may be value-free in the sense of prizing fairness and seeking 
the discipline of a balanced approach, but it is an art, or a set of arts characterized by moral 
and intellectual virtues antithetical to a strict rules-for-their-own-sakes mentality, or an 
arbitrary or indoctrinational, obedience-oriented mentality of teaching. While there cannot 
be a moral test for who should teach, it could be explained from on's college coursework 
on through teacher training just what virtues and attitudes make it possible to listen, respond 
to and cultivate students' critical thinking capacities. 

VI. Conclusion, This year's NAEP reports on reading and writing contain noteworthy 
findings which ought to encourage men and women cfgood will , since some recent reforms 
seem lo have stuck, several generations of students have been \\tn to improve as positive 
changes were introduced and to retain th benefits of those changes from 4th through to the 
llth grade. Impressive gains have been made for black and hispanic students in several 
reading areas, and males and some minorities have improved, relatively, in their writing. 
We would be in danger of scuttling successful ideas if these findings were ignored. 
Though these positive achievements take place at low levels of achievement, and are chiefly 
concerned with lower-order skills of th use of the mind, thy are vital, basic and must be 
retained. 

But in the area of constructive criticism based on deplorably low proficiency scores 
in reading and writing, my primary generalization would be that our children are not being 
presented with challenges or coaching in the higher-order skills, or, it may be, are being 



^ . E.g. in Critical Thinking. What Every Person Needs to Survive in a Rap idly Changing 
Worid (Rohnert Park: Center for Critical Thinking and Moral Critique, 1990), chapters 13 
and 20 (with Joel Rudinow), inter alia . 
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presented a confused hodge-podge derived from the noise of current educationist or 
philosophers' writings on the topic.^ The basics seem to be clear: the students cannot 
follow passages, or write passages, containing several ideas or pieces of evidence; they 
cannot draw inferences on their own, and cannot spot inferences when drawn by writers or 
speakers; they cannot see connections, trace out consequences, ask questions fo. needed 
information, or critique claims by rebuttal, or propose claims at a high level of persuasion. 
They cannot paraphrase the gist of a passage in their own words, and often cannot spot a 
paraphrase when presented, lliey tend to fold, or quit on a task, when it gets past a single 
item, or a known step, or the carrying out of a single instruction. Paradoxically, though 
these findings seem to suggest we have come upon a mindless generation fit only for 
obedience and the lowest levels of consciousness, our knowledge of the human gene pool 
and the worldwide distribution of higher cortical functions suggests that the fault lies, not 
in the students, but in the culture as a habitat for the growth of the mind. Educators are not 
in control of that culture - far from it - but we face a huge task, and probably not an 
unlimited amount of time to do it in, if we a^e to change curricula and our own moral and 
intellectual capacities in order to be able to give tnem what they, and we, need. 



. Regrettably, but again anecdotally, this is confirmed at a higher level in my own 
experience, when taking part in recruiting, reading files and taking part in interviews to hire 
instructors in critical thinking for a required college class. Here, too, what is called 'critical 
thinking' is a confusing melange, ranging from the old wine in new bottles (of using formal 
logic but in new clothing), to the unripe, unbottled juices of pure miscellanies, with some 
partially-coherent treatments in the middle. Though some works seem to me to stand out 
as beacons - chiefly Michael Scriven's Reasoning * it is too soon in the critical thinking 
movement for philosophers and educators to have reached agreement among themselves, 
much less with each other, as to what one would need to learn in this area, and in what 
order. 



